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DTI0  QUAHW  IHSl 


Polyhedral  Methods  for  the  Maximum  Clique 

Problem 


Abstract 

This  paper  presents  an  integer  programming  approach  to  the  maximum 
clique  problem.  An  initial  linear  programming  relaxation  is  solved  and, 
when  there  is  an  integrality  gap,  this  relaxation  is  strengthened  using  one 
of  several  tightening  procedures.  This  is  done  through  the  addition  of 
cutting  planes  to  the  linear  program.  The  bulk  of  the  paper  deals  with 
theoretical  and  computational  issues  associated  with  the  generation  of 
these  cuts.  In  particular,  we  describe  how  to  obtain  cuts  from  the 
positive  semi-definiteness  of  an  underlying  matrix.  The  various  cuts  are 
then  compared  in  a  computational  experiment.  These  cuts  can  be 
incorporated  into  a  branch-and-cut  algorithm  and  we  report  results  with 
such  an  algorithm  on  some  of  the  DIMACS  benchmark  instances. 
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1  Introduction 


A  clique  in  a  graph  G  =  {V.,E)  is  a  vertex  set  C  C  V  such  that  every  two  vertices  of  C  are 
adjacent.  The  clique  polytope  C{G)  is  the  convex  hull  of  the  incidence  vectors  of  the  cliques 
in  G: 

C{G)  =  conv{x  €  {0,1}”  :  Xi  +  xj  <  l,V(*,j)  ^  E}, 

where  n  =  |y|.  The  maximum  weight  clique  problem  consists  of  finding  max{wx  :  x  G 
C{G)}.  The  linear  programming  relaxation  oi  C{G) 

FC{G)  =  {a;  e  :  Xi  +  Xj  <  1, V(f,j)  ^  E} 

is  sometimes  called  the  fractional  clique  polytope.  This  polytope  strictly  contains  C{G) 
whenever  G  is  not  the  complement  of  a  bipartite  graph,  and  there  are  several  well  known 
classes  of  inequalities  which  are  valid  for  C{G)  but  violated  by  points  in  FC{G).  The 
addition  of  some  of  these  inequalities  to  FC{G)  gives  rise  to  stronger  relaxations  of  C{G). 
A  vertex  set  S  is  stable  if  every  two  vertices  in  S  are  nonadjacent.  Clearly,  for  any  stable 
set  S  of  size  greater  than  two,  <  1  is  valid  for  C{G)  and  strengthens  the  inequalities 

Xi  +  X j  <  1  of  FC{G)  for  all  i,j  €  S. 

Our  approach  in  this  paper  is  to  strengthen  the  relaxation  FC{G).  We  start  with  a  very 
simple  strengthening, 

SC{G)  =  {x  e  ^  X.-  <  1,  fork  =  l,...m} 

ieSk 

where  the  sets  Sk,  k  =  are  maximal  stable  sets  with  the  property  that,  for  any 

{i,j)  ^  E,  there  exists  at  least  one  k  such  that  i,j  €  Sk-  Constructing  such  a  family  Sk, 
k  =  1, ...  ,m,  is  done  using  a  straightforward  greedy  procedure.  The  focus  of  the  paper  is 
on  procedures  that  achieve  further  strengthenings  in  a  systematic  fashion.  We  illustrate  how 
some  recent  results  on  lift-and- project  tightening  procedures  for  mixed  0-1  programs  can  be 
applied  to  relaxations  of  the  clique  polytope  such  as  SC{G). 

In  Section  2  we  describe  three  different  lift-and-project  tightening  procedures:  the  first 
one  is  the  procedure  of  Balas,  Ceria  and  Cornuejols  [1]  and  is  the  simplest  of  the  three, 
while  the  other  two  can  be  interpreted  as  variations  of  tightening  procedures  of  Lov^z 
and  Schrijver  [6] .  In  all  three  cases  the  problem  is  formulated  in  a  higher  dimensional  space, 
where  the  resulting  relaxation  can  be  tightened  by  imposing  the  0-1  condition  on  one  or  more 
variables.  The  tightened  formulation  is  then  projected  onto  the  space  of  original  variables. 
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In  Section  3,  we  show  how  to  generate  cutting  planes  that  arise  from  these  tightening 
procedures.  We  can  generate  the  “deepest”  cut  of  the  family  by  solving  a  linear  program. 

In  Section  4,  we  explain  how  the  higher  dimensional  formulations  can  be  further  strength¬ 
ened  by  requiring  positive  semidefiniteness  of  a  certain  matrix  defined  by  the  newly  intro¬ 
duced  variables.  This  idea  is  also  related  to  a  tightening  procedure  of  Lovasz  and  Schrijver 
[6],  different  from  those  mentioned  above. 

In  Section  5,  we  show  that  finding  a  deepest  cut  for  the  positive  semidefinite  cuts  requires 
the  solution  of  a  semi-infinite  linear  program  where  the  entering  column  is  found  by  an 
eigenvalue  computation. 

Finally,  in  Section  6,  we  report  on  our  computational  experience.  The  first  part  of  the 
computational  study  investigates  the  quality  of  the  cuts  obtained  from  the  various  tightening 
procedures.  In  the  second  part,  we  apply  MIPO  to  some  of  the  DIMACS  benchmark  in¬ 
stances:  MIPO  is  a  general  purpose  mixed  integer  program  optimizer  based  on  the  tightening 
procedure  of  [1]. 


2  Lift-and-project  tightening  procedures 

Before  we  can  discuss  the  algorithms  used  in  our  computational  study  to  strengthen  relax¬ 
ations  of  the  maximum  clique  problem  in  the  x-space  of  vertex  variables,  we  need  to  lift 
the  problem  into  a  higher  dimensional  space.  Indeed,  the  tightening  procedures  are  easily 
understood  in  that  higher  space.  This  section  discusses  several  options  for  the  tightening 
step.  A  projection  step  is  then  used  to  return  to  the  x-space.  We  delay  until  Section  3  the 
discussion  of  algorithms  for  finding  a  deepest  cut  from  this  projection. 

Let  K  he  &  relaxation  of  the  clique  polytope  C{G)  such  that  C{G)  =  conv{K^),  where 
Ar°  =  {x€{0,l}”:x€/'f}.  We  represent  the  inequalities  defining  K  by  the  system  Ax  >  b. 

2.1  The  basic  lift-and-project  procedure 

The  following  results  are  from  Balas,  Ceria  and  Cornuejols  [1],  and  are  also  valid  for  more 
general  pure  and  mixed  0-1  programs.  Consider  a  fixed  j  £  {1, . . .  ,n}. 
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Procedure  1 


1.  Multiply  Ax  >b  with  1  —  xj  and  Xj  to  obtain  the  nonlinear  system 

(1  —  Xj){Ax  —  b)>0 

Xj{Ax  —  b)  >  0.  (1) 

2.  Linearize  (1)  by  substituting  yi  for  XiXj,  z  =  i  ^  j,  and  Xj  for  xj.  Call  the 

polyhedron  defined  by  the  resulting  system  Mj{K). 

3.  Project  Mj{K)  onto  the  x-space.  Call  the  resulting  polyhedron  Pj{K). 

The  calculation  of  the  projection  used  in  Step  3  amounts  to  eliminating  the  variables  yi  by 
taking  certain  specific  linear  combinations  of  the  inequalities  defining  Mj{K).  Even  though 
the  system  Mj{K)  is  small  (the  number  of  its  nonzero  entries  is  of  order  n),  computing  the 
full  projection  Pj{K)  is  computationally  prohibitive  in  general.  In  Section  3,  we  will  show 
how  to  obtain  useful  inequalities  defining  Pj(K). 

It  is  not  hard  to  see  that  C  Pj{K)  C  K.  In  fact,  Pj{K)  has  the  following  considerably 
stronger  properties: 

Theorem  2.1  Pj{K)  =  conv{K  fl  {x  :  Xj  €  {0, 1}}). 

For  t  >  2,  define  Pii,...,u{K)  =  Pu{Pn,...,u-iiK))- 

Theorem  2.2  For  any  t  E  {1,. . .  ,n}, 

Pii,...,it{^)  =  conv{K  n  {a:  :  Xj  €  {0, 1}  for  j  =  ii, . . . 

Corollary  2.3  Pi . niK)  =  conv{K°). 

Theorem  2.2  implies  the  following. 

Property  2.4  For  i  ^  j, 

conv{conv{K  fl  {x  :  x,-  G  {0,1}})  H  {x  :  Xj  G  {0,1}}) 

=  conv[conv{K  H  {x  :  Xj  G  {0, 1}})  fl  {x  :  Xj  G  {0, 1}}) 

=  conv{K  n  {x  :  Xi,Xj  G  {0,1}}). 
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Theorem  2.2  says  that  if  Procedure  1  is  applied  repeatedly,  each  time  with  respect  to  a 
different  variable  xj,  for  j  €  S,  the  outcome  is  conv{x  e  K  :  Xj  G  {0, 1}  for  j  G  -S'}.  In  other 
words,  the  ordering  of  ii, . . . , it  in  the  definition  of  P^,...,it{K)  is  irrelevant.  This  justifies  the 
simpler  notation  Ps{K).  A  further  result  (see  Sherali  and  Adams  [7]  and  also  [1])  states  that 
if  Step  1  of  Procedure  1  is  applied  repeatedly  with  respect  to  several  variables  Xj,  for  j  G  S, 
followed  by  a  single  linearization  and  projection,  the  outcome  is  the  same  as  if  Procedure  1 
had  been  applied  repeatedly  in  its  entirety.  To  be  specific,  consider  the  following  procedure 
for  a  given  S  C  {!,...,  n). 

Procedure  1’ 

1.  Multiply  Ax>b  with  flieSi  HjeSzll  -a^j)  for  every  partition  {Si,  S2)  of  S,  to  obtain 
the  nonlinear  system 

n  a:,  n  (1  -  ^1)  -  ^)  ^0,  VSi,S2  c  -S  s.t.  SiUS2  =  s,  SinS2  =  0.  (2) 

i€Si  jeS2 

2.  Linearize  (2)  by  substituting  Xj  for  Xj,  p  >  2,  and  yw  for  IW'l  >  2.  Call  the 

polyhedron  defined  by  the  resulting  system  Ms{K). 

3.  Project  Ms{K)  onto  the  x-space.  Call  the  resulting  polyhedron  Ps{K). 

We  then  have 

Theorem  2.5  Ps{K)  =  Ps{K)- 

For  proofs  of  these  theorems,  as  well  as  their  connection  with  earlier  work  and  further 
details,  the  reader  is  referred  to  [1]. 


2.2  Stronger  tightening  procedures 

In  this  section,  we  use  other  inequalities  (different  from  (1  — Xj)  >  0  and  Xj  >  0)  in  the  multi¬ 
plication  step  of  lift-and-project  tightening  procedures.  This  idea  was  introduced  by  Lovasz 
and  Schrijver  [6].  For  a  detailed  description  of  this  and  other  lift-and-project  tightening 
procedures  the  reader  is  referred  to  Ceria  [4]. 

Given  any  stable  set  S,  the  inequality  x(,S')  <  1  is  valid  for  C{G).  The  modified  lift-and- 
project  tightening  procedure  is  defined  as  follows: 
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Procedure  2 


1.  Multiply  Ax  >  b  with  1  —  x{S)  and  xi,  i  €  S,  to  obtain  the  nonlinear  system 

(1  —  x(5'))(Ax  —  6)  >  0 

Xi(Ax  —  b)  >  0,  i  ^  S.  (3) 

2.  Linearize  (3)  by  substituting  ya  for  XjXf,  ^  =  1, . . .  ,n,  i  £  S,  and  ar,-  for  xf,  i  E.  S,  and 
setting  yu  =  yu.  Call  the  polyhedron  defined  by  the  resulting  system  Hs(K). 

3.  Project  Hs{K)  onto  the  x-space.  Call  the  resulting  polyhedron  Rs{K). 

Appying  Procedure  2  amounts  to  convexifying  over  all  the  variables  in  S  as  shown  by 
the  following  theorem. 

Theorem  2.6  Rs{K)  =  conv{K  fl  {a:  :  a;,-  €  {0, 1},  for  all  i  6  S'}). 

Proof:  We  claim  that  applying  Step  1  of  Procedure  2,  then  carrying  out  the  multipli¬ 
cations  in  (3)  and  substituting  a;,-  for  x?,  i  G  S',  yields  the  same  nonlinear  system,  say  (*), 
that  we  obtain  by  applying  Step  1  of  Procedure  1’  with  respect  to  S',  carrying  out  the  mul¬ 
tiplications  in  (2)  and  substituting  x,-  for  xf,  p  >  2.  Then  the  theorem  follows  since  the 
linearization  and  projection  of  (*)  onto  the  x-space  yields  conv(Kr\{x  :  x,-  G  {0, 1},  Vi  G  5}) 
by  Theorem  2.5. 

To  prove  the  claim,  notice  that  after  carrying  out  the  multiplications  in  (2)  and  substi¬ 
tuting  Xi  for  xf,  i  G  S',  p  >  2,  all  those  terms  containing  the  product  XjXj  for  some  pair 
i,j  G  S'  become  0  since,  for  each  such  pair,  1  —  x,—  xj  >  0  is  part  of  the  system  Ax  —  b>0, 
and  x,(l  —  Xi  —  Xj)  >  0  implies  — x.Xj  >  0.  Thus  the  only  inequalities  of  (*)  with  nonzero 
terms  are 

•  the  inequality  corresponding  to  the  partition  {Si,  S'2)  with  S'!  =  0  and  S2  =  S  which  is 

Ax  —  b  —  ^(Axxi  —  bxi)  >  0. 

t'es 

•  the  |5|  inequalities  corresponding  to  partitions  of  the  form  (S'!,  S'2)  with  Si  =  {i},  S2  = 
S  \  {z},  which  are 

Axxj  —  bxi  >  0,  for  i  G  S. 
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But  these  are  precisely  the  inequalities  obtained  by  executing  the  multiplications  in  (3), 
which  proves  the  claim.  □ 


Let  T  be  a  set  of  vertices  of  G  and  let  5  =  {^i, . . . ,  5'^}  be  a  cover  of  the  vertices  of  T 
by  stable  sets  Sj  C  T.  When  5  is  a  partition,  this  is  a  fc-coloring  of  G{T),  but  in  general  we 
will  not  choose  to  be  a  partition.  For  2<t<k,  define  Rsy,...,St{K)'=  Rst{Rsi,...,St-i{K)). 

Theorem  2.7  For  any  t  ^  .  ,k}, 

Rs, . s,{K)  =  conv{K  n  {x  :  x,-  €  {0, 1},  Vi  G  5'i  U  . . .  U  5'i}). 

Proof:  By  induction  on  t.  For  t  =  1,  the  statement  is  just  Theorem  2.6  applied  to  5  =  ^i. 


Suppose  the  statement  is  true  for  f  =  1, . . .  ,p  —  1  and  let  t  =  p.  Then 

Rs. . sAK)  =  RsARs, . v.(^))  (4) 

=  Rsp{conv{K  n  {x  :  x,-  €  {0, 1},  Vi  €  -Si  U  ...  U  ^p-i}))  (5) 

=  conv{conv{K  H  {x  :  x^  6  {0, 1},  i  G  U  . . .  U  5'p_i}) 

n{x:  x,  G  {0,1},  ViG^p})  (6) 

=  conv{K  n  {x  :  x,-  G  {0, 1),  Vi  G  /Si  U  . . .  U  Sp})  (7) 


where  (4)  follows  from  the  definition,  (5)  from  the  induction  hypothesis,  (6)  from  Theorem  2.6 
and  (7)  from  Property  2.4.  □ 

Corollary  2.8  =  conv{K  D  {x,-  G  {0,1},  i  G  T}). 

Proof:  Follows  directly  from  Theorem  2.7.  □ 

Applying  the  previous  result  with  T  =  V  implies  that  we  can  obtain  C*(G)  in  a  number 
of  steps  equal  to  the  cardinality  of  any  stable  set  cover  of  the  vertices  of  G.  Of  course,  this 
number  is  bounded  below  by  the  chromatic  number  of  G.  In  a  random  graph  with  edge 
probability  of  1  /2,  one  can  obtain  a  stable  set  cover  of  size  in  the  order  of  n/  loga  n  with 
probability  1  —  o(l). 

It  is  possible  to  further  generalize  these  results  by  allowing  for  multiplication  with  in¬ 
equalities  corresponding  to  more  than  one  stable  set  in  S.  In  particular,  we  are  interested 
in  the  following  extension: 
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Procedure  3 


1.  Multiply  Ax  >  b  with  1  —  x{Sj),  j  —  1, . . . ,  and  Xi,  i  ^  T  to  obtain  the  nonlinear 
system 

{1  -  x{Sj)){Ax  -  b)  >  0,  j  =  l,...,k 

Xi{Ax  —  b)  >  0,  z  G  T.  (8) 

2.  Linearize  (8)  by  substituting  yn  for  XiX(,  £  =  i  ^  T,  and  Xi  for  xf,  i  G  T, 

and  setting  yu  =  yu-  Call  the  polyhedron  defined  by  the  resulting  system  Hi,,,k{K). 

3.  Project  onto  the  x-space.  Call  the  resulting  polyhedron  Ri_,,kiK). 

The  special  case  where  T  =  V  and  each  Sj  reduces  to  a  single  vertex  of  G,  say  Sj  =  {i}, 
is  one  of  the  tightening  procedures  proposed  by  Lovasz  and  Schrijver  [6].  When  T  ^  V 
and  Sj  =  {j}  for  j  G  T,  let  us  denote  by  Nt{K)  the  polyhedron  obtained  in  Step  3  of 
Procedure  3.  It  is  easy  to  see  that,  for  any  S, 

Ri.AK)  C  Nt{K) 

and,  in  fact,  the  relaxation  Ri,„k{^)  can  be  substantially  stronger  when  some  of  the  Sj's  in 
S  have  cardinality  greater  than  one.  For  example,  when  K  =  FC{G),  Theorem  2.6  implies 
that 

Ri...k{K)  C  C\^j_^conv{K  n  {x  :  x,-  G  {0, 1}  for  all  i  G  Sj}) 
whereas  it  can  be  shown  that 

Nt{K)  =  f)j^TConv{K  n  {x  :  Xj  G  {0, 1}}). 


3  Lift-and-Project  Cutting  Planes 

In  this  section  we  address  the  generation  of  cutting  planes  for  the  maximum  clique  problem 

max{wx  :  x  £  K,  Xj  ^  {0, 1},  j  =  1, . . .  ,n},  (9) 

where,  as  before,  K  =  FC{G)  or  any  valid  relaxation  of  C{G).  The  cut  generation  algorithms 
presented  in  this  paper  work  indifferently  for  the  maximum  cardinality  or  maximum  weight 
clique  problems. 
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3.1  Cut  generation  linear  program  for  Procedure  1 


First,  we  consider  Procedure  1  discussed  in  Section  2.  Balas,  Ceria  and  Cornuejols  [1] 
showed  that  in  order  to  generate  a  valid  inequality  ax  >  ^  for  Pj{K)  that  cuts  off  x  (the 
current  solution  to  the  linear  programming  relaxation  of  (9))  by  the  largest  amount,  where 
0  <  <  1,  we  need  to  solve  the  following  linear  program; 

max  uAx  +  uoXj 
subject  to 

uA  —  vA-\-  (uo  —  “^0)^3  = 
uh  = 
vh-\-vo  = 
u^v  > 

Substituting  vq  =  —vh  —  1  into  uAj  —  vAj  +  uq  ~  fo  =  0?  then  substituting 

uo  =  -uAj  +  vAj  -vb-1 
=  -uAj  +  vAj  +  ub-vb 

into  the  objective  function,  yields  the  linear  program 


0 

-1 

-1 

0. 


(10) 


max  —  u{{b  —  Aj)xj  +  Ax)  +  v{b  —  Aj)xj 
subject  to 

(— «  +  v)Ai  =  0  i  ^  j  (11) 

ub  =  —1 
u,v  >  0, 


where  A{  denotes  the  column  of  A.  Then  the  cut  ax  >  ^  is  obtained  as  follows: 


a,  =  uAi  i  ^  j 

aj  =  ub  +  v{Aj  —  b) 

P  =  ub=  —1. 
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3.2  Cut  generation  linear  programs  for  Procedures  2  and  3:  an 
illustration 

Now  we  discuss  the  generation  of  cutting  planes  for  the  stronger  tightening  procedures  of 
Section  2.  They  require  solving  linear  programs  whose  size  depends  on  the  sets  S  or  T.  We 
illustrate  the  cases  where  5  is  a  stable  set  of  size  2  (Procedure  2)  and  T  is  a  clique  of  size  2 
(Procedure  3). 

Suppose  S  =  where  {j,£)  ^  E  m  Procedure  2.  A  deepest  cut  (i.e.  an  inequality 

ax  >  13  valid  for  Rs{K)  which  cuts  off  x  by  the  largest  amount)  is  obtained  by  solving  the 
linear  program: 

max  —  u{{b  —  Aj)xj  +  Ax)  +  v^(b  —  Aj)xj  —  u((b  —  Ai)x(,  +  Ax)  +  v^{b  —  Ai)x(, 
subject  to 

(— u  +  v^)A(,  +(— u  +  v^)Aj  =  0 

{—u  +  v^)Ai  =  0  (12) 

{-uAv^)Ai  =  0  i^j,i 

ub  =  —1 

u,v^,v^  >  0 

and  defining 

cti  =  uAi 

aj  =  ub  +  v^{Aj  —  b) 
a(  =  ub  +  v^{A(  —  b) 

13  =  ub=  —1. 

The  linear  programs  (11)  and  (12)  optimize  over  the  sets  conv{K  fl  {a:  :  Xj  G  {0,1}}) 
and  conv{K  fl  {x  :  Xj,Xi  G  {0,1}}),  respectively.  The  corresponding  inequalities  ax  >  ^ 
belong  to  the  family  of  disjunctive  cuts,  to  which  the  strengthening  procedure  of  Balas  and 
Jeroslow  [3]  can  be  applied. 

Now  let  T  =  {i,^},  where  {j,i)  G  E.  We  consider  Procedure  3  where  =  {j}  and 
S2  =  {£}.  The  deepest  cut  ax  >  l3  is  obtained  by  solving  the  linear  program: 
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max  —u^{{b 

—  Aj)xj  +  Ax)  +  v^{b 

subject  to 

(— u-’  +  v^)Ae 

4-(-«^  +  v^)Aj  = 

(— +  v^)Ai 

= 

{-u^  +  v^)Ai  = 

u^b 

+u^b  = 

(13) 


u\v\u  ,v 


>  0 


and  defining 


aj  =  {u^  —  v^)b  +  u^Aj 

at  —  {u^  —  v^)h  +  u^At 

^  =  u^h  +  u^h  =  —2. 

The  linear  program  (13)  is  significantly  larger  than  (11)  but  it  is  closely  related  to  it  and 
this  relationship  can  be  used  to  avoid  solving  (13)  from  scratch.  Indeed,  assuming  that  we 
have  solved  (11)  for  two  different  fractional  components  of  x,  say  j  and  £,  we  can  combine 
the  resulting  optimum  solutions  to  obtain  a  feasible  solution  to  (13).  Then,  using  sensitivity 
analysis,  one  can  improve  upon  the  combined  solution  by  noting  that  the  difference  between 
linear  program  (13)  and  the  two  versions  of  (11)  is  two  pairs  of  aggregated  constaints,  namely 
=  0  and  {—u^  +  v^)Aj  =  0  on  the  one  hand,  and  u^b  =  —1  and  u^b  =  —1  on 
the  other  hand.  Increasing  the  RHS  of  one  constraint  in  the  pair  by  6  in  one  linear  program 
and  decreasing  it  by  6  in  the  other  will  improve  the  solution  of  (13)  at  a  rate  given  by  the 
difference  of  the  corresponding  dual  variables  in  (11).  The  updated  u\  u^,  yield  a 
stronger  cut  in  general,  although  not  necessarily  the  strongest  for  (13). 

The  linear  programs  (12)  and  (13),  just  like  (11),  can  be  restricted  to  the  space  of 
fractional  components  in  x,  and  the  resulting  cuts  can  be  lifted,  as  explained  in  [1]. 
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3.3  Cut  generation  linear  program  for  Nt(K) 


In  this  section,  we  give  the  linear  program  that  needs  to  be  solved  for  finding  a  deepest  cut 
which  is  valid  for  Nt{K).  Remember  that  Nt{K)  arises  from  Procedure  3  by  imposing  the 
disjunctions  =  0  V  Xj  =  1). 

We  start  with  the  higher  dimensional  relaxation  that  arises  in  Step  2  of  Procedure  3.  It 
contains  the  constraints 


—z^ 

-y3 

= 

0 

Az^  —bzl 

> 

0 

z^- 

= 

0 

Ay^ 

-ho 

> 

0 

y] 

-yi 

0 

^0 

Wo 

1 

for  all  j  €  T,  together  with  the  symmetry  constraints 

yi-Vj  =  0 

for  all  j,k  £  T  such  that  j  <  k.  Any  valid  inequality  ax  >  ^  for  Nt{K)  is  obtained  from 
this  system  by  eliminating  the  y  and  z  variables.  This  yields: 


Eier 

— 

a 

Q,j  —u^A 

= 

0, 

Vjer 

a^  —v^A 

-V^Cj 

= 

0, 

VjeT 

u^b 

> 

Vj€T 

v^b 

w 

> 

V;  €r 

Ei€T 

> 

> 

0. 

If  we  eliminate  the  a^  ’s,  we  can  write  the  cut  generation  linear  program  for  finding  the 
deepest  valid  cut  for  Nt{K)  as  follows. 
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Max  ^  u^{—Ax)  +  ^  ul{—Xj) 
jeT  3€T 


k>j 


i<j 

0, 

Vjer 

u^b 

> 

VjeT 

v^h  -f  Vq 

> 

VjGT 

Eft 

ler 

> 

> 

0. 

Note  that  it  is  possible  to  solve  this  linear  program  by  restricting  it  to  the  subspace 
corresponding  to  the  fractional  components  of  x.  Indeed,  it  can  be  shown,  using  an  argument 
similar  to  that  in  [1],  that  the  solution  obtained  in  this  subspace  can  be  lifted  optimally  into 
a  deepest  cut  for  the  full  space. 


4  Positive  Semi-Definite  Constraints 

In  this  section  and  the  next,  we  show  how  to  further  strengthen  the  polytope  Nt{K).  For 
ease  of  notation,  we  present  the  results  for  T  =  V.  Once  again,  we  first  derive  the  results  in 
the  higher  dimensional  space. 

The  idea  of  using  a  positive  semi-definite  (psd  for  short)  constraint  in  higher  dimensional 
formulations  of  0-1  integer  programs  appeared  first  in  [6]  :  if  in  Step  2  of  Procedures  2  or  3 
all  the  XiXi  products  appear  1  <  «,^  <  n,  then  it  is  a  valid  constraint  to  enforce  psdness 
of  the  symmetric  matrix  Y  substituted  for  xx^.  In  the  case  where  K  =  FC{G),  the  higher 
dimensional  relaxation  M{K)  can  be  substantially  strengthened  by  psdness.  In  this  section, 
we  show  how  to  construct  a  closely  related  family  of  constraints. 

We  denote  psdness  of  a  matrix  A  by  writing  A^O.  Let  Y  be  the  symmetric  matrix  of 
our  variables  yu,  diag  Y  =  x,Yo  some  n  x  p  matrix  formed  from  elements  oi  Y,  p  <  n,  and 
5X0  some  fixed  p  x  p  matrix.  We  consider  the  following  psd  constraint 

Y  -  YoSYj^  h  0.  (14) 

The  separation  problem  for  (14)  is  to  check  whether  (14)  holds  for  given  matrix  Y  and,  if  it 
does  not,  to  produce  a  cutting  plane  which  is  valid  for  all  matrices  satisfying  (14)  but  not 
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for  the  given  matrix.  In  the  following  we  show  that  (14)  behaves  well  algorithmically  and 
we  give  two  choices  for  S  and  Yq  with  the  property  that  the  resulting  constraint  (14)  is  valid 
for  our  higher  dimensional  representations. 

Theorem  4,1  The  separation  problem  for  (If)  can  be  solved  in  polynomial  time. 

Proof.  Suppose  we  are  given  Y  and  let  Yq  be  some  n  x  p  matrix  formed  from  elements  of 
Y,  p  <  n.  Since  5  ^  0  we  can  write  S  =  Z^Z  for  some  matrix  Z.  Then  Y  satisfies  (14)  if 
and  only  if 


u^{Y  -  Fo5Yo^)u  >  0  for  all  u  €  |u|  =  1  (15) 

i.e. 

Y  0  uvF  —  \ZYou\^  >  0  for  all  u  €  |u|  =  1  (16) 

where  o  denotes  the  inner  product  of  matrices.  For  a  moment,  let  us  fix  u,  and  let  a  =  ZYqu. 
An  elementary  calculation  shows  that 

—  |a|^  =  min{2A^a  +  |Ap  :  A  G  3^”}  (17) 

where  the  minimum  is  attained  for  A  =  —a.  Thus  (16)  is  equivalent  to 

min{F  o  uu'^  +  2A^(FFou)  +  |A|2  :  A  €  >0  for  all  u  G  3?'^,  |n|  =  1  (18) 

which  is  the  same  as 

Youu'^  +  2A^(ZFou)  +  |Ap  >  0  for  all  u  G  3«",  |u|  =  l,for  all  A  G  (19) 

Thus,  (19)  is  a  linearized  form  of  (14),  and  separation  goes  as  follows: 


1.  Find  u  G  |«|  =  1,  s.t.  u^(F  —  YoSYo)u  <  0  .  If  no  such  u  exists,  F  —  b  0. 

This  step  is  of  order  since  it  amounts  to  performing  Gaussian  elimination,  see  [5] 
page  295. 

2.  If  u  exists,  find  A  s.t.  2X^(ZYou)  +  |Ap  =  |(Zio«)P  (  i-e.  A  =  —{ZYqu)  ).  Then  F 
violates  (19)  with  the  above  A  and  u. 
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Our  first  choice  for  S  and  Yq  which  makes  (14)  valid  for  the  higher  dimensional  relaxations 
is  S  =  I,  Yo  —  X.  Thus  we  obtain  the  constraint 


Y  —  xx^  y  0 


(20) 


This  constraint  can  be  expressed  in  a  different  way,  related  to  a  construction  of  Lovasz  and 
Schrijver. 


Proposition  4.2  Let  Y  he  given,  diag  Y=x, 

Y.= 

Then  Y  —  xx^  hO  if  and  only  ifY^.  y  0. 
Proof.  Define  the  n  by  n  matrix  L  as 

L  = 


1  X 
X  Y 


1  0 
-X  I 


Since  L  is  of  full  rank,  clearly  K»  is  psd  if  and  only  if  LY»L'^  is  psd.  However,  it  is  easy  to 
check  that 


0 


^0  Y-xx^  )' 

thus  the  proposition  follows.  • 

The  matrix  K  is  related  to  the  work  of  Lovasz  and  Schrijver:  in  [6],  they  homogenize  their 
lower  dimensional  formulation  Ax  >  b  into  Ax  —  bxo  >  0.  Thus  by  multiplying  variables  they 
obtain  ann  +  lbyn  +  1  matrix  having  its  first  row  and  column  equal  to  its  diagonal.  The 
top  left  element  in  the  matrix  is  xo  =  1.  Imposing  psdness  on  this  matrix  thus  is  equivalent 
to  our  constraint  (20). 

Our  second  choice  for  S  and  io  is  iS  =  Yq  =  Y ,  where  Bx  <  c  is  a 

system  of  inequalities  valid  for  K ,  such  that  both  B  and  c  have  nonnegative  components. 

Proposition  4.3  Constraint  (14)  with  the  above  choice  of  S  and  Yq  is  satisfied  byY  —  xx 
if  X  is  a  0-1  vector  satisfying  Bx  <  c. 
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Proof. 


V  -  YoSYo 


(21) 

(22) 


=  xx^  —  {l/\c\^)xx^  Bxx'^ 

=  (i-(i/icni5xiv^ 


Bx  <  c  implies  |Bxp  <  |cp,  thus  the  last  expression  is  indeed  psd.  ■ 

A  simple  observation  shows  that  we  can  use  the  psd  constraint  (14)  in  a  restricted  way:  if 
in  our  higher  dimensional  formulations  we  do  not  have  all  possible  variables  yu,  1  <  i,£  <  n, 
i.e.  we  only  generated  some  elements  of  the  nxn  matrix  F,  all  we  must  do  is  enforce  psdness 
of  the  available  principal  submatrices. 


5  Generating  Cutting  Planes  from  Semi-Definite  Con¬ 
straints 

In  this  section  we  address  the  question  of  generating  cuts  in  the  x-space,  based  on  the  positive 
semi-definiteness  arguments  developed  in  Section  4. 

Consider  a  higher-dimensional  representation  of  our  0-1  program 


Dx  -f  By  >  c  (23) 

where  the  linear  inequalities  arise  from  Step  2  of  Procedures  3  and  from  the  valid  constraints 
for  the  convex  set  defined  by  (14).  Note  that  system  (23)  has  infinitely  many  constraints. 
Nevertheless,  for  the  sahe  of  simplicity,  we  use  the  above  matrix  notation. 

The  components  t/,j  of  the  vector  y  can  be  viewed  as  elements  of  a  square  matrix  Y  with 
diagonal  x.  It  was  shown  in  Section  4  that  the  separation  problem  for  the  semi-definite 
constraints  in  (23)  amounts  to  computing  a  minimum  eigenvalue.  In  our  computational 
experiments,  we  use  the  following  version  of  the  separation  algorithm. 

1.  Given  (x,  F),  find  the  smallest  eigenvalue  of  Y  —  xx^.  If  it  is  nonnegative,  F  —  xx^  y  0. 
Otherwise,  let  u  €  9?”  be  a  corresponding  nonzero  eigenvector. 

2.  An  inequality  of  (23)  violated  by  (x,  F)  is  (F  —  xx^)  o  vv^  >  0. 
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Suppose  that  we  would  like  to  generate  a  cutting  plane  ax  >  ^  in  the  space  of  the  x 
variables  from  system  (23).  In  the  rest  of  the  section  we  show  that  the  separation  algorithm 
will  suffice  to  generate  the  columns  of  our  cut-generation  LP;  as  a  by-product,  we  also  get  a 
nice  interpretation  of  its  dual. 

Assume  that  we  are  given  a  current  fractional  point,  x.  To  derive  a  cutting  plane  of  the 
form  ax  >  /?  (with  a  <  0  and  /3  <  0)  which  cuts  off  x  by  the  largest  amount,  we  must  solve 

max  —ax  +  ^ 
s.t.  a  —  uD  =  0 

uB  =0  (24) 

—uc-\-^  <  0 

u  >  0. 

Suppose  that  we  normalize  our  system  by  choosing  ^  =  —1.  Next,  a  can  be  eliminated 
from  (24)  to  get 

max  u{—Dx) 
s.t.  uB  =  0 

(25) 

—uc  <  1 

u  >0. 

Consider  the  dual  of  (25), 

min  t 

s.t.  t  >  0  (26) 

By  —  ct  >  —Dx 

which  can  be  rewritten  as 

min  t 

s.t.  t  >  0  (27) 

Dx  -|-  By  >  ct. 

Comparing  (23)  and  (27),  we  can  interpret  (27)  as  follows:  Since  we  are  able  to  generate 
a  cut  violated  by  x,  there  cannot  be  a  y  with  (x,y)  satisfying  (23).  Hence  the  system 

Dx  -\-  By  >  ct  is  infeasible  for  t  =  1,  and  the  minimum  of  t  in  (27)  is  attained  for  some 

t*  >  1.  The  larger  the  value  of  T,  the  larger  the  amount  -(ax  -h  1)  by  which  the  point  x  is 
cut  off  by  the  inequality  ax  >  -1.  Thus  we  have  the  following  duality  relationship: 
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Theorem  5.1  {  The  maximum  amount  by  which  ax  >  —1  cuts  off  x  equals  1  plus  the 
minimum  value  of  t  for  which  the  system  Ax  +  By  >  ct  becomes  feasible. 

Also,  (27)  makes  it  clear  how  to  generate  a  cut,  even  if  (23)  is  only  given  by  a  separation 
oracle.  Finding  a  column  with  negative  reduced  cost  in  (25)  can  be  done  by  constructing  a 
ducil  vector  {t,y),  then  checking  whether  {t,y)  satisfies  the  constraints  of  (26).  This  can  be 
done  using  the  same  separation  algorithm  as  for  (23). 


6  Computational  Experience 

The  purpose  of  this  section  is  twofold:  First,  we  want  to  investigate  the  strength  of  our 
various  cuts,  in  particular  those  obtained  by  using  positive  semi-definiteness.  Second,  we 
want  to  see  how  a  general-purpose  mixed  integer  programing  code,  using  Procedure  1  in  a 
branch-and-cut  framework,  performs  on  the  maximum  clique  problem.  We  report  on  our 
computational  experience  with  several  instances  from  the  testbed  provided  in  the  DIMACS 
Challenge. 

All  experiments  were  performed  starting  from  formulations  FC{G)  or  SC{G)  described 
in  Section  1.  The  stable  sets  Sk,  A:  =  1, . . . ,  m,  covering  the  edges  of  G  in  the  definition  of 
SC{G)  are  generated  using  a  greedy  procedure.  In  the  generic  step  of  the  procedure,  called 
GENSTABLE,  we  have  a  family  S  =  {^i, . . . ,  Sk-i}  of  stable  sets.  If  {i,j)  ^  E  and  no  stable 
set  of  S  contains  both  i  and  j,  initialize  Sk  =  {iij}-  Among  the  vertices  of  G  which  have 
no  neighbor  in  Sk,  select  one  with  lowest  degree  in  G,  add  it  to  Sk  and  repeat  until  Sk  is  a 
maximal  stable  set.  Add  Sk  to  S. 


6.1  Comparison  of  the  cuts 

Before  comparing  alternatives  for  generating  cuts,  we  first  give  a  feel  for  the  strength  of  the 
higher  dimensional  relaxations  MffK),  Hs{K)  and  Hi,„k{K).  We  illustrate  the  difference 
on  the  test  problem  hamming6-4. 

The  gap  closed  by  Procedure  1  was  ^628-4^  ^  closed  by  Procedure  2  (jS'l  =  4) 

was  «  9%  and  that  closed  by  Procedure  3  (|r|  =  4)  was  «  17%.  In  all  cases 

the  choice  of  variables  was  based  on  their  fractionality:  we  observed  (and  it  is  supported  by 
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theoretical  evidence  [4],  page  94)  that  using  components  of  the  current  solution  which  are 
close  to  1/2  usually  gives  the  greatest  improvements  in  Procedures  1-3.  This  example  shows 
that  substantial  gains  can  be  achieved  by  using  the  stronger  formulations,  and  Procedure  3 
seems  the  most  attractive. 

We  conducted  three  experiments  with  Procedure  3  to  generate  cutting  planes.  In  all 
cases  we  chose  k  =  6,  Sj  =  {iy},  j  =  and  T  =  That  is,  each  of  the  k 

stable  sets  we  multiplied  by  to  obtain  our  higher  dimensional  representation,  consisted  of  a 
single  vertex.  The  above  choice  of  k  is  justified  as  follows:  it  should  be  large  enough,  so  that 
positive  semi-definiteness  can  make  a  difference;  and  small  enough  so  that  the  cut-generation 
linear  programs  can  still  be  solved  in  a  reasonable  amount  of  time. 

Our  test  set  contained  six  randomly  generated  graphs,  three  with  30  vertices  and  20,  30 
and  40  %  density,  respectively,  and  three  with  50  vertices  and  15,  20  and  25  %  density. 

Our  first  experiment  was  done  without  using  semi-definiteness  in  the  cut-generation. 
When  we  used  FC{G)  as  our  initial  relaxation,  in  every  case  we  obtained  a  cut  that  could 
be  obtained  with  Procedure  1  with  some  {ij}  for  j  €  {l,...,fc}.  These  results  are  not 
surprising:  As  mentioned  in  Section  2,  when  K  =  FC{G), 

NxiK)  =  f)j^T(^onv{K  D  {x  :  Xj  €  {0,1}}). 

Since  our  procedure  generates  facets  of  Nt(K)  and  the  facets  of  Nt{K)  are  facets  of  conv^KO 
{x  :  Xj  €  {0,1}})  for  some  j  €  {1,...,A;},  they  are  also  obtainable  by  using  Procedure  1. 
When  we  used  SC{G)  as  our  initial  relaxation,  we  still  observed  the  same  behavior  in  most 
cases. 

In  our  second  experiment  we  used  semi-definiteness.  First,  we  solved  our  initial  LP- 
relaxation.  Then  we  chose  our  6  variables  T  to  be  as  fractional  as  possible,  and  in  a  way 
that  the  corresponding  vertices  induced  a  dense  subgraph.  Our  experience  was  that  semi¬ 
definiteness  did  not  improve  our  cuts  in  any  of  the  cases,  no  matter  whether  our  initial 
relaxation  was  FC{G)  or  SC{G). 

In  our  third  experiment  we  also  used  semi-definiteness.  Again,  first  we  solved  the  initial 
LP-relaxation.  Then  our  6  variables  T  were  chosen  to  be  as  fractional  as  possible,  and  such 
that  the  corresponding  variables  induced  a  sparse  subgraph.  Our  experience  is  that  semi¬ 
definiteness  substantially  improved  our  cuts,  when  our  initial  relaxation  was  FC{G).  In 
Table  1  we  show  the  improvement  in  the  depth  of  the  generated  cuts,  averaged  over  4  cuts. 
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Vertices 

Density 

Depth  without  PSD 

Depth  with  PSD 

30 

20 

0.5 

0.516 

30 

30 

0.5 

0.791 

30 

40 

0.5 

1.0 

50 

15 

0.5 

0.58 

50 

20 

0.5 

0.732 

50 

25 

0.5 

0.903 

Table  1:  Improvement  in  cut-depth  using  positive  semi-definiteness 


when  our  initial  relaxation  was  FC{G).  (The  depth  of  a  cut  ax  >  /3  is  the  euclidean  distance 
between  the  point  x  it  is  designed  to  cut  off,  and  the  hyperplane  ax  =  ^).  With  our  initial 
relaxation  chosen  as  SC{G),  the  improvements  were  marginal,  or  nil. 

We  summarize  the  results  of  this  section  as  follows.  Substantial  gains  can  be  achieved  by 
using  tightening  Procedure  3  as  compared  to  Procedure  1,  as  is  examplified  by  our  results 
on  hamming6-4.  Unfortunately,  these  encouraging  results  do  not  translate  into  substantially 
better  cuts.  In  other  words,  even  though  the  set  Nt{K)  is  a  tighter  relaxation  than  Pj{K), 
the  facets  of  Nt{K)  are  often  not  better  than  those  of  Pj{K)  when  used  individually  as  cuts. 
The  situation  improves  somewhat  with  positive  semi-definite  cuts  but  not  enough  to  justify 
the  substantial  extra  computing  time.  As  a  consequence,  we  settled  on  the  cuts  generated 
from  Procedure  1  for  our  more  extensive  computational  study. 

6.2  Results  with  Procedure  1 

The  lift-and-project  cutting  planes  associated  with  Procedure  1  are  incorporated  into  a 
branch-and-cut  algorithm.  The  resulting  code,  called  MIPO,  can  solve  general  mixed  0-1 
programs,  and  is  described  in  [2].  In  order  to  apply  it  to  the  maximum  clique  problem  we 
first  use  the  heuristic  GENSTABLE  described  earlier  to  generate  a  cover  of  the  nonedges  by 
stable  sets.  Then  we  apply  MIPO  to  this  formulation.  The  results  are  shown  in  Table  2. 
Times  are  in  seconds  on  an  Apollo  desktop  workstation  HP720. 

The  following  observations  can  be  made  from  this  table. 
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For  the  instances  which  have  a  large  maximum  clique,  MIPO  did  relatively  well.  Specifi¬ 
cally,  for  the  nine  instances  in  Table  2  which  have  a  maximum  clique  of  size  greater  than  40, 
only  two  instance  took  more  than  800  seconds.  On  the  other  seven  instances,  MIPO  only 
took  an  average  of  210  seconds.  This  is  in  contrast  to  the  behavior  of  several  other  exact 
algorithms,  which  tend  to  deteriorate  when  the  size  of  the  maximum  clique  increases. 

In  40  %  of  the  instances,  GENSTABLE  generated  a  formulation  with  no  integrality  gap, 
resulting  in  excellent  computing  times.  For  the  problems  where  cuts  were  required,  the 
computing  times  were  significantly  greater.  But  remember  that  MIPO  is  a  general  purpose 
mixed  integer  program  optimizer.  We  did  not  taylor  it  in  any  way  for  the  maximum  clique 
problem. 

One  aspect  of  this  class  of  integer  programs  is  that  LP-relaxations  tend  to  have  highly 
fractional  solutions.  As  a  result,  we  can  rarely  take  advantage  of  the  lifting  step  described  in 
[2],  where  the  cuts  are  generated  in  the  space  of  fractional  variables  of  the  current  solution, 
then  lifted  into  the  whole  space. 

It  is  clear  from  Table  2  that  the  cuts  are  working:  only  five  problems  required  more  than 
300  cuts  and  nodes  in  the  branch-and-cut  tree.  The  main  drawback  is  the  computing  time 
required  to  generate  the  cuts.  Thus  there  is  a  need  to  address  the  question  of  finding  faster 
cut-generation  heuristics.  An  attractive  direction  for  future  research  would  be  to  generate 
cutting  planes  in  subspaces  of  variables  defined  by  relatively  small  subsets  of  vertices,  instead 
of  the  full  space  as  is  currently  the  case  with  MIPO. 
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Problem 

name 

Vertices 

Edges 

Density 

Clique 

Size 

MIPO 

Cuts 

MIPO 

Branches 

CPU 

Time 

brock200-2 

200 

9876 

50  % 

12 

5140 

8746 

28000 

c-fat200-l 

200 

1534 

8% 

12 

120 

80 

730 

c-fat200-2 

200 

3235 

16% 

24 

20 

2 

370 

c-fat200-5 

200 

8473 

43  % 

58 

60 

46 

2000 

C125.9 

125 

6975 

90  % 

34 

1640 

3244 

2400 

gen200-p0.9-44 

200 

17910 

90  % 

44 

174 

240 

670 

gen200-p0.9-55 

200 

17910 

90  % 

55 

0 

0 

10 

hamming6-2 

64 

1824 

90  % 

32 

0 

0 

0.13 

hamming6-4 

64 

704 

35  % 

4 

75 

54 

16 

hamming8-2 

256 

31616 

97  % 

128 

0 

0 

1.5 

hamming8-4 

256 

20864 

64  % 

16 

78 

100 

1500 

johnson8-2-4 

28 

210 

56  % 

4 

0 

0 

0.01 

johnson8-4-4 

70 

1855 

77  % 

14 

0 

0 

0.15 

johnsonl6-2-4 

120 

5460 

76  % 

8 

0 

0 

0.09 

keller4 

171 

9435 

65  % 

11 

2398 

4104 

6100 

p-hat300-l 

300 

10933 

24  % 

8 

4921 

9466 

49000 

rand 100. 90 

100 

4455 

90  % 

35 

56 

72 

27 

rand 100. 95 

100 

4703 

95  % 

52 

0 

0 

0.14 

rand200.95 

200 

18905 

95  % 

70 

2889 

4822 

7500 

san200-0.7-l 

200 

13930 

70  % 

30 

0 

0 

11 

san200-0.7-2 

200 

13930 

70  % 

18 

56 

38 

280 

san200-0.9-l 

200 

17910 

90  % 

70 

0 

0 

2.6 

san200-0.9-2 

200 

17910 

90  % 

60 

0 

0 

7.8 

san200-0.9-3 

200 

17910 

90  % 

44 

279 

246 

780 

san400-0.5-l 

400 

39900 

50  % 

13 

39 

18 

4400 

san400-0.7-2 

400 

55860 

70  % 

30 

216 

162 

26000 

Table  2:  Runs  obtained  using  the  cuts  of  Procedure  1 

♦ 
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